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SYSTEM, APPARATUS, METHOD AND designing telecommunication systems. It also required time 

ARTICLE OF MANUFACTURE FOR to gather the panel of people, administer the tests, and 

EVALUATING THE QUALITY OF A formulate the resultant scores. 

TRANSMISSION CHANNEL USING VOICE Early analog telecommunication systems would send 

RECOGNITION TECHNOLOGY 5 au dio tones across an audio channel to measure the received 

signal, noise and distortion (SINAD). Although this method 
was objective and repeatable, it became obsolete with the 

CROSS-REFERENCE TO RELATED advent of digital communications technologies. Digital com- 

APPLICATIONS munications implement vocoders that digitally represent 

"Not Applicable" 10 yo ^ co according to a set of voice characteristics. Since 

SINAD uses audio tones as the source, vocoders distort the 

STATEMENT REGARDING FEDERALLY si S nal m< * produce erroneous results. 

SPONSORED RESEARCH OR DEVELOPMENT Modem voice quality measurement solutions use the 

Perceptual Voice Quality Measure (PSQM) algorithm 

"Not Applicable" is defined m ITTJ Recommendation P.861. Although the PSQM 

REFERENCE OF A "MICROFICHE APPENDIX" f 1 ^" * accurate for many telecommunication 

technologies, it has some shortcomings that make it difficult 

"Not Applicable" to implement. Tools that use the PSQM algorithm must be 

calibrated using MOS testing. Since MOS testing is timely 

BACKGROUND 20 and expensive, it is difficult for test tool manufacturers using 

i r- u c T PSQM to quickly react to market needs. Furthermore, 

1. Field of Invention DC ^ U . ,?„- . , . . , 7 Jn . . 

PSQM is difficult to implement m VoIP systems because 

The present invention relates to systems analysis and large amounts of latency disrupt the synchronization of the 

more particularly, to a system, apparatus, method and article reC eived signal to the undistorted reference signal, 

of manufacture for evaluating the quality of a transmission 25 

channel using voice recognition technology. Among other SUMMARY OF THE INVENTION 

features to be described in more detail later, the system, ^ t referred t0 as 0 pera™, 

apparatus, method and article of manufacture is particularly Ufli { ^ yoice re mtion technology for £ pur . 

suited for providing an objective measurement, in real-time, of measuri the voice m of ^communication 

of the quality of audio signals transmitted along the trans- 30 s ^ ^ ^ pme nt of telecommunication 

mission channel. ,ui • • _c i * . * i *i_ * 

technology, engineers require powerful test tools that can 

2. Description of the Related Art measure the performance of their system and compare that 
Many technologies are used in today's telecommunica- performance to other competing systems. The present inven- 
tion systems, so powerful test tools are essential for 35 tion provides an accurate practical tool that measures voice 
designing, building and optimizing such systems. Engineers quality in real-time and records audio for later examination, 
require tools that measure technology-specific information Because objective voice quality scores are universal mea- 
as well as overall quality of service. In industries such as surements that apply to all telecommunication systems, The 
wireless telecommunication and Voice Over IP (VoIP), present invention can be used for a wide variety of testing 
intense competition has driven companies to provide better applications. 

and better quality at lower end-user cost. Test tools are The present invention measures the voice quality of 
needed that can objectively measure the quality of service telecommunication systems in a way that is accurate, 
(QoS) for differing telecommunication technologies while repeatable, and practical for real-time implementation. The 
providing a common benchmark criteria for all. present invention resolves the shortcomings of previous 
The best way of measuring telecommunication QoS is to 45 methods while providing the flexibility for different lan- 
measure the actual quality of transmitted voice. Voice qual- guages and different technologies. The present invention can 
ity measurement can be subjectively measured using a panel also be used on high latency technologies such as those 
of people but this can result in inconsistent results that are implemented with VoIP. 

not repeatable. This method can also be expensive when The key to the present invention's flexibility involves a 
used for subsequent trials, such as when engineers may take 50 unique application of voice recognition technology for the 
the results of the panel test, modify the network for better purpose of measuring voice quality. Voice recognition corn- 
performance, and need to repeat the testing cycle. Engineers bined with high quality audio equipment and some custom 
need a way of objectively measuring voice quality that is software development creates a powerful and effective solu- 
accurate, repeatable, economical, and consistent across dif- tion. 

ferent telecommunication technologies. 55 Therefore, in accordance with one aspect of the present 

Telecommunication is international. Since many telecom- invention there is provided a method for evaluating the 

munication vocoders are often designed to the operate for a quality of a transmission channel having a first port and a 

particular language or voice type, QoS might digress if used second port, comprising the steps of: transmitting a first 

by an international user base. Voice quality measurement signal from said first port to said second port via said 

tools must be the operational for different languages and 60 transmission channel; receiving said first signal at said 

voice types. second port; analyzing said first signal using voice recog- 

The first solutions for measuring voice quality were nition technology to generate a quality score within a 

techniques of mean opinion score (MOS). Panels of people predetermined time frame; monitoring step (c) to ensure that 

were used in a controlled environment to give an opinion of said quality score is generated during said predetermined 

samples of voice quality. The scores from each person were 65 time frame; assigning a default quality score if a quality 

then averaged to give a resulting MOS score. This solution score is not generated during said predetermined time frame; 

was not repeatable, costly, and not practical for engineers transmitting a second signal from said second port to said 
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first port, via said transmission channel, after said quality The present invention can easily adapt to different commu- 

score is either generated or assigned said default quality nication standards and applications by modifying the audio 

score; and monitoring step (f) to ensure that said second interface coupled to the communication device. For illus- 

signal is not being transmitted to said first port while said trative purposes, however, what follows describes one appli- 

first signal is being processed. 5 cation of the present invention to wireless technology. 

In accordance with a second aspect of the present 2. Configuration of a System Constructed in 

invention, there is provided a method further comprising the Accordance with the Present Invention 

step of storing, in a storage medium, the first signal and the FIG. 1 is a physical diagram representing a system 

quality score. constructed in accordance with the present invention that 

In accordance with a third aspect of the present invention, 10 provides for a means of analysis at both a mobile and a 

there is provided a method further comprising the step of landline side. FIGS. 2 and 3 are functional diagrams of 

displaying said quality score alternative embodiments of a system constructed in accor- 

In accordance with a third aspect of the present invention, d » nce J** "j 6 Z™ al tion . tha / for a means 

there is provided a method wherein said quality score is „ ° f "S^ " '7™' ' Slde - 

d* 1 ed n 1 t' Figures describe the novel and inventive aspects of the 

displayed in reai-time. present invention. 

These aspects and other aspects, features and advantages Generally speaking, the system comprises one or more 

of the present invention will become better understood with data processing systems running one or more software 

regard to the following description, accompanying drawings applications. The data processing systems are electronically 

and appended claims. 2 o coupled to one or more communication devices via a com- 
munications medium. 

BRIEF DESCRIPTION OF THE DRAWINGS A. Hardware Components 

c _ , , t f . u * * At tne mobile side, communication devices 135 and 140 

Exemplary emboduBents of the present mvention are now are deyices ^ mterface ^ communications medium to 

briefly described with reference to the following drawings: ^smit and/or receive audio signals. As shown, an example 

FIG. 1 is a physical diagram representing a system of such a device is a wireless telephone. The analog audio 

constructed in accordance with the present invention. signal that is produced by the communication device is 

FIG. 2 is a functional diagram of one embodiment of a usually line level audio but might vary depending on the 

system constructed in accordance with the present invention. tv P e of communication device. 

FIGS. 3A and 3B show a functional diagram of an 30 * the - side ;™ m * u f* tion ^ evices 170 and180 

nUa i- t r t . . . are typically devices that interface with a communications 

**ZtZ r t \ ^wr^T a£ medi ™ t0 < ransm * ™™ «xKo signals. As shown, 

dance with the presen tinvenUon (FIGS. 3 A and 3B l consti- afl fc of sucfa a deyice fe a ^ ^ Tel hone ' 

tute one figure, with FIG. 3A being the left side and FIG 3B Network (psTN) device> which ^ a 

vfc \\ CDtire ^ heremafter referred t0 PSm The interface device automaticaUy answers incoming 

as FIG. 3). 35 ca jjg £ ac jj i n t er f ace device is designed for the appropriate 

FIG. 4 is flow diagram of primary operational steps communication medium it interfaces with (i.e.: some appli- 

performed by one embodiment of a system constructed in cations of VoIP, radio-to-radio telecommunication, etc.) 

accordance with present invention. Situated at both the mobile and landline side, are data 

processing systems 115, 145. Date processing systems 115, 

DETAILED DESCRIPTION OF THE 40 145 are typically standard computers designed according to 

INVENTION widely understood principles. As illustrated, the computers 

In the following detailed description of the embodiments, °? mftise a prOCeSSOr ' mem0ry 12 , 5 ' "'""f^ 

„ f „ . j . a. • j i_- . Wlt ° the processor, e.g., storage media, and selected 

reference is made to the accompanying drawings which M . u 1 1 • * a . ♦ a • u 

, u^ an f u- u - u u f peripherals, namely input and output devices, such as, a 

form a part hereof, and in which is shown by way of _„V, rt A \m iin a a- 1 a • ine ha 1 

;n„o*«t*r u a ' * • u- u *u ■ \- 45 sound card 130, 160 and a display device 105, 110, also 

illustration specific embodiments in which the mvention . . . , ' * J 9 9 

, j t*u u a- a *i_ a * associated with the processor. 

may be practiced. These embodiments are described in a a iia • j * » . 

a * m « ui + u 1 *ii a • *t. ^ * Th e sound card 130, 160 is a device having a stereo 

sufficient detail to enable those skilled in the art to practice , , 4 . , . , 4 , , . , 4 , 

*- a ' * l. a * a ii_ 4. it • j • channel that is used to convert analog audio signals to digital 

the invention, and it is to be understood that other embodi- a- • ^ ai • TL & °T. 

_ , _ , ' .,. j * .* . . 1 1 • i j 1 audio signals and/or vice-versa. The audio device sampling 

ments may be utilized and that structural, logical and elec- „ . K . . in MC , TT *\ . & 

. * o! . u j %u . j * *u • ** 50 rate in our current system is 11.025 kHz, and is 

trical changes may be made without departing from the spirit ^ > ^ 

and scope of the present inventions. The following detailed ur .» A A f J^' , , * , ; u ' 

* *u r « « k ♦ 1 v PCMCIA card, USB sound device, etc. In the present 

description is, therefore, not to be taken in a limiting sense, ,u u 1 • ^ • 

nnA iU „ ' . ac a i t. invention, the mono channel communication devices or 

and the scope of the present inventions is defined only by the psTN imerfaces _ a _ h OTnnccted to a stereo cnannel via 

appended claims. The leading digit(s) of the reference <c interlaces areeacn connected to a stereo cnannel via 

„r.™k„«- ;„ *u„ n* 11 . j . *u « 55 a physical adapter. The present mvention splits the incoming 

numbers in the Figures usually correspond to the figure t „ u 1 ■ * * i_ ^ J ^ ^ 

1 . „« # u*u *• .u »-j *• 1 l stereo channel mto two mono channels and passes that to the 

number, with the exception that identical components which • ... , , . ~ . 

„ A • _ 1.- 1 £ 'a 4*c a i_ jl r voice recogmtion module or application. This allows two 

appear in multiple figures are identified by the same refer- . , . 4 . ^ t , 

ence numbers communication devices to be used on one stereo sound card 

jack. FIG. 1 shows the stereo sound card interfacing with 

1 Overview 60 two coniniunication devices. However, there can be 1 ... N 

communication devices (where N is a positive value), lim- 

The present invention is specifically designed to measure ited only by the number of sound cards (as shown in the 

the voice quality of a communication device's audio chan- present system there are two communication devices per 

nel. It measures both the uplink audio (mobile to landline) sound card) and the processor's capability, 

and downlink audio (landline to mobile). To completely 65 Communications medium 190 (see FIG. 2) can be a 

isolate the testing of uplink and downlink audio, audio choice of a PSTN, a wireless network, a satellite commu- 

samples are alternated and analyzed in a half duplex fashion. nications network, or some other communications network. 
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B. Software Components 

Associated with the processor of the computer 115 and 
145 and executable by the processor are several program- 
ming modules or software objects, namely a voice recogni- 
tion application 120Aand a quality measurement application 5 
120B. Both the voice recognition application 120Aand the 
quality measurement application 120B cooperate with each 
other and are central to the operation of the present inven- 
tion. 

Voice recognition application 120A is a voice recognition 
software module or engine (e.g., see SR engine 230 in FIG. 10 
3) that produces a preliminary score or confidence factor 
corresponding to voice quality. A single, phonetically bal- 
anced word phrase (audio bite) is set up as the grammar (i.e.: 
"Jump the fence and hurry up the bank"). The voice recog- 
nition engine's confidence factor is based on how closely the 15 
incoming digitized audio matches the known phrase. Labo- 
ratory testing has found that some, but not all voice recog- 
nition engines allow one phrase in a grammar to affect other 
distinct phrases in the same grammar. A single grammar 
phrase is suggested in order to minimize this effect. 20 

Quality measurement application 120B is a programming 
module, that in cooperation with the voice recognition 
module 120A is the heart of the present invention. As shown 
in FIGS. 2 and 3, Quality measurement application 120B 
comprises several cooperating functions and data files 25 
namely, Audio Storage 250, VQ Score Calculation 240, VQ 
Score Storage 270, VQ Score Display 295, VQ Score Sync 
290, Output Trigger 280, Wave Player 260, and Audio 
Replay Control 285. 

Audio Storage 250 is a programming module that stores 30 
the digital audio for later review. 

VQ Score Calculation 240 is a programming module that 
converts the score or confidence factor 
(SRConfidencefactor) generated by the voice recognition 
engine to an objective voice quality score (VQScore) 35 
between 1 (bad) and 5 (excellent) using the following 
mathematical equation: 

Vj2Score-(5WConfidencefactor) m dM-dB 

where dM and dB are calibration factors. 40 

The equation is defined during laboratory calibration. 
Each unique sample sentence transmitted by a transmitting 
communication device and received by a corresponding 
receiving communication device has a different calibration. 
Once the linear calibration factors (dM for slope and dB for 45 
y-intercept) are determined, the programming module cal- 
culates the voice quality score using the above equation. 

VQ Score Storage 270 is a programming module that 
stores the voice quality scores for later review. 

VQ Score Display 295 is a programming module that 50 
displays the voice quality scores to the user in real-time via 
a display device. The word "display device" is used generi- 
cally here and can encompass a variety of solid state devices 
or means such as, a CRT display, a LCD display, a printed 
report, a voice notification means and/or any other like 55 
device or mechanism. 

VQ Score Sync 290 is a programming module that 
monitors all communication devices and outputs a synchro- 
nization pulse when incoming audio has been scored. This 
programming module ensures that outgoing audio is not sent 60 
when incoming audio is being processed. It also detects if 
any or all of the communication devices have not received 
audio in a given time (missing pulse detection based upon a 
sliding time window). 

Output Trigger 280 is a programming module that starts 65 
output wave audio when triggered by the VQ Score Sync 
programming module. 290. 
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Wave Player 260 is a programming module that plays 
outgoing audio samples. 

Audio Replay Control 285 is a programming module that 
allows a user to replay stored digital audio as it corresponds 
to any voice quality score. 

FIG. 2 is a functional diagram of one embodiment of a 
system constructed in accordance with the present invention. 
Here, the system is constructed to provide half duplex voice 
quality measurement of an audio channel. At the landline 
side a compact disc, a mini disc or other audio output device 
205 provides audio signals to a communication device or an 
interface device 210. The communication device or an 
interface device 210 transmits the audio signals via an audio 
channel 190 where it is received by a communication device 
or an interface device 210 at the mobile side for processing 
at the mobile side in accordance with the principles of the 
present invention. 

FIG. 3 is a functional diagram of an alternative embodi- 
ment of a system constructed in accordance with the present 
invention. Here, the system is constructed to provide full 
duplex voice quality measurement of an audio channel. At 
both the landline side and the mobile side audio signals are 
processed in accordance with the principles of the present 
invention. 

3. Method of Using a System Constructed in 
Accordance with the Present Invention 

FIG. 4 is flow diagram of the primary operational steps of 
one embodiment of a system constructed in accordance with 
present invention. 

A. Downlink Process 

At 400, a user, using a computer located at the landline 
side, accesses the system. At 405, a seven (7) second clock 
is triggered and downlink analog audio begins to play 
(landline to mobile). The clock is tuned using any widely 
known sliding window algorithm to minimize the time 
between uplink/downlink round trips. At 410, the downlink 
analog audio is received by a mobile unit where it is 
digitized. At 415, the now digitized audio is sent to the voice 
recognition engine module where a confidence factor is 
generated by the voice recognition engine optionally, the 
digitized audio can be stored in a data file for later retrieval 
and processing. At 420, a voice quality score is calculated 
based upon the confidence factor generated during the 
previous step. At 425, the voice quality score is sent to the 
VQ Sync module. Preferably, the score is displayed on a 
display device in real-time and/or stored in a data file for 
later retrieval and processing. At 430, when the VQ Sync 
module receives a score from all communication devices — 
i.e. telephony or network interface devices, it sends a 
synchronization pulse to the audio output device. If a 
communication device does not detect an audio sample, a 
default value of 1 or "bad" is scored. 

B. Uplink Process 

At 435, uplink audio is played to each communication 
device connected to the system. At 440, the uplink audio is 
received by the landline unit and digitized. At 445, the 
digitized audio is sent to the voice recognition engine and 
saved to storage. At 450, a voice quality score is calculated 
based upon the confidence factor generated during the 
previous step. At 455, the voice quality score is sent to the 
VQ Sync module 290. Preferably, the score is displayed on 
a display device 295, in real-time, and/or stored in a data file 
for later retrieval and processing. At step 460, when the VQ 
Sync module 290 receives a score from all communication 
devices — i.e. telephony or network interface devices, it 
sends a synchronization pulse to the audio output device. If 



08/26/2004, EAST Version: 1.4.1 



US 6,700, 

7 

a communication device does not detect an audio sample, a 
default value of 1 of "bad" is scored. 

4. Operation Steps in More Detail 

Referring to FIGS. 2 and 3, the two parts of the present 5 
invention, the mobile unit and the landline unit, are shown. 
Both units are essentially the same and can be adapted for 
other types of telecommunication applications, such as, 
VoIP. In a general sense, the mobile unit and landline unit 
work together like two people having a conversation. First, 10 
the landline unit transmits a short voice sentence that the 
mobile unit measures for phonetic accuracy and scores 
according to a scale similar to MOS. Next, the mobile unit 
transmits a short voice sentence and the landline unit mea- 
sures the quality. This conversation continues as long as the 15 
call is up. 

At both the mobile unit and landline unit, an audio device 
in a personal computer is used to convert analog signals to 
digital signals or digital signals to analog signals. The 
parameters of audio conversion are as follows: 2 0 

Sample type-16 bit mono PCM 

Sample rate=11.025 kHz 

This sample type is consistent with the audio bandwidth 
of most telecommunication systems. To accurately capture 
analog audio in digital format, it is generally accepted to 25 
take a 2-3 times sampling rate of the audio bandwidth. Most 
telecommunication systems have an audio channel of about 
3 kHz. 11.025 kHz is 3.675 times 3 kHz. 11.025 kHz is also 
the sample type used by most voice recognition engines. 

Most telephony/communication devices have an audio 30 
interface that can be connected to external devices. This 
interface usually uses line level or microphone level signals. 
The audio devices in the mobile unit and landline unit are 
line level. Some system configurations require audio adapt- 
ers to interface the audio devices to the telephony/ 35 
communication devices. Because most professional audio 
equipment has far better quality than telephony/ 
communication technology, distortion caused by such equip- 
ment is negligible. Most of the distortion measured by the 
present invention comes from the telephony/communication 40 
audio channel. This is precisely the distortion engineers 
would like to measure. Received audio that has been digi- 
tized by the audio device is sent to the voice recognition 
engine and also saved as an audio file, such as a standard 
Microsoft wave format audio files. 45 

In a preferred embodiment the stereo card supports two 
telephony/communications devices. Each device is a mon- 
aural device connected to either the left or right channel of 
the stereo sound card jack via an adapter. The present 
invention splits the stereo audio data into two mono chan- 50 
nels during recording from the telephony/communication 
device, or merges two mono audio channels to a stereo 
channel for playback to the telephony/communication 
device. 

In one embodiment of the present invention, the voice 55 
recognition module is L&H ASR 1600® voice recognition 
engine. Other voice recognition engines, such as the 
Microsoft SR Engine® and IBM Via-Voice® can also be 
used. Whatever the specific engine chosen, to get the voice 
recognition engine to properly measure voice quality, it must 60 
be properly setup and configured as follows: 

1) A grammar must be defined to contain only one user 
phrase (or "word" as the voice recognition software refers to 
it, even though it can be multiple spoken words such as 
"Jump the fence and hurry up the bank**). 65 

2) The user phrase must be trained using the audio sample 
used for voice quality measurement. 



,953 Bl 

8 

3) The audio sample used to train the user phrase should 
be 2.5 seconds and must be professionally recorded with 
minimal background noise and clear voice. We suggest a 
phonetically balanced phrase. 

4) The language supported by the voice recognition 
engine must be consistent with the language of the audio 
sample. 

5) The parameters of the voice recognition engine must be 
set to include all erroneous noise and distortion in the 
calculation of the confidence factor. Most voice recognition 
engines produce a confidence factor score that represents the 
engine's confidence in a sentence it has detected. 

6) The engine must be set to continually monitor received 
audio. 

When the voice engine detects the audio sample, it reports 
the confidence factor to the voice quality score calculation 
object. The voice quality score is calculated using param- 
eters that were calibrated through laboratory experimenta- 
tion. Calibration is performed by sending known audio 
samples into the voice engine and examining the confidence 
score reported. After many sentences are sent to the voice 
engine, a mathematical equation can be created to translate 
confidence factors into voice quality scores. This math- 
ematical equation is then always used with its corresponding 
audio sample. 

A software component called the VQ Sync 290 is used to 
trigger the playing of output audio only after the audio 
sample from all telephone devices has been measured. The 
VQ Sync 290 has a sliding time window in which it expects 
to receive all the voice quality scores. When all scores have 
been received, it triggers the playing of output audio. If a 
score is not received for a device within the expected time 
window, the trigger will be sent and a score of 1.0 will be 
recorded for the audio sample. Output audio is played using 
the same audio device configuration as the received audio. 

Each of the voice quality scores is tagged with a time/date 
stamp. In our current system, we use a resolution of 1 
millisecond. This can be used to correlate with data from a 
global positioning system (GPS) to plot the voice quality 
scores on a map, thus allowing an engineer to see where the 
voice quality degrades, to plot coverage maps, etc. 

After a test session is completed, the recorded audio can 
be replayed using the Audio Replay Control module. This 
module makes it easy for engineers to examine the actual 
audio samples measured by the present invention. There are 
many types of audio distortion and the voice quality score 
alone cannot represent this distortion fully. By listening to 
the audio, engineers can better understand the cause of poor 
audio quality. 

In our trials, we have found that the present invention 
provides a voice quality test solution that is highly adaptable 
and practical. For instance, it can be used for a wide variety 
of telephony/communication technologies while providing 
accurate, economical, repeatable measurements of voice 
quality. We have tested with at least six different brands of 
wireless phones, four different telephone network operator 
systems (i.e.: phone networks), numerous corded and cord- 
less phones, and even audio recording and playback devices, 
such as compact discs players and mini disc players. None 
of these device or network changes required major modifi- 
cations to the present invention. The only required modifi- 
cation would be the interface to the new audio device. 

The present invention's real-time performance, versatility 
and ability to record audio makes it a great, tool for 
engineers working on telephony/communication systems 
since feedback time is minimized in a test/get results/tune 
telephony — communication device or network/test cycle. 
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Furthermore, the present invention's ability to compare and 
contrast the performance of multiple technologies at once in 
a time-synced fashion provides engineers with the tools they 
need to benchmark their systems against competing systems. 

5. Advantages and Closing 5 

The present invention is fundamentally different from 
existing solutions. First, the present invention can use 
readily available and affordable off-the-shelf professional 
audio devices to convert analog/digital audio. Other sohi- 10 
tions use specialized and expensive Digital Signal Process- 
ing equipment to convert analog/digital audio. Second, the 
present invention does not use audio synchronization signals 
to mark the beginning of an audio sample. Other solutions 
use audio synchronization signals that become problematic 
with low rate vocoders or VoIP technologies. Third, the 
present invention records and stores audio for later exami- 
nation. Other solutions do not record audio. Fourth, the 
present invention instantly adapts to new communication 
devices. Other solutions must be individually calibrated for 
every communication device used. Fifth, the present inven- 20 
tion supports multiple languages as defined by the particular 
voice recognition engine capabilities. Other solutions only 
work with English. Sixth, the present invention synchro- 
nizes the measurement of multiple communication devices 
operating simultaneously. Other solutions do not synchro- 25 
nize multiple devices thus any derived measurements tend to 
be several seconds out of sync. 

The present invention has numerous advantages over 
existing solutions, chief of which include the following: 1) 
accurate measurements of voice quality for many types of 30 
distortion. Test methods are immediately repeatable and 
consistent. There is no need to have a panel listen to audio 
recordings each time a test is necessary. In addition, if there 
are different people in the panel each time, the results are 
inconsistent; 2) calibration is no longer required for every 35 
communication device used. If the communication device 
has a way to output audio, it can be hooked to the system. 
There is no need to modify the present invention for differ- 
ing codecs or hardware, unlike existing systems; 3) engi- 
neers can evaluate new communication devices and tech- 
nologies (such as W-CDMA or even prototype phones) the 40 
instant they are available. All existing voice quality mea- 
surement solutions require huge lead-times for integration of 
the new communication device or technology; 4) real-time 
voice quality measurement. For example if the present 
invention is used to test a phone network for eight hours, 45 
eight hours worth of voice quality scores are generated at the 
end of the testing cycle. With certain existing solutions, at a 
minimum, another eight hours is required to have people 
listen to the recordings and render their voice quality scores; 
5) engineers can add audio quality measurement to tradi- 50 
tional field measurement tools allowing them to see how 
factors such as poor signal strength affect audio quality; 6) 
engineers can compare the performance of their system to 
competitors' systems to see where more optimization is 
required; using the present invention gives engineers the 
perspective they need by showing them how consumers 
judge audio quality and network performance; and 7) the 
present invention allows two communication devices per 
each stereo sound card jack, doubling the number of com- 
munication devices that can be measured simultaneously. 

Having now described one or more preferred embodi- 60 
ments of the invention, it should be apparent to those skilled 
in the art that the foregoing is illustrative only and not 
limiting, having been presented by way of example only. All 
the features disclosed in this specification (including any 
accompanying claims, abstract, and drawings) may be 65 
replaced by alternative features serving the same purpose, 
equivalents or similar purpose, unless expressly stated oth- 
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erwise. Therefore, numerous other embodiments of the 
modifications thereof are contemplated as falling within the 
scope of the present invention as defined by the appended 
claims and equivalents thereto. 
What is claimed is: 

1. A method for evaluating the quality of a transmission 
channel having a first port and a second port, comprising the 
steps of: 

(a) transmitting a first signal from said first port to said 
second port via said transmission channel, said first 
signal being an analog signal; 

(b) receiving said first signal at said second port, and 
converting said first signal to a digital signal; 

(c) analyzing said digital signal using voice recognition 
technology to generate a quality score within a prede- 
termined time frame; 

(d) monitoring step (c) to ensure that said quality score is 
generated during said predetermined time frame; 

(e) assigning a default quality score if a quality score is 
not generated during said predetermined time frame; 

(f) transmitting a second signal from said second port to 
said first port, via said transmission channel, after said 
quality score is either generated or assigned said default 
quality score; 

(g) monitoring step (f) to ensure that said second signal is 
not being transmitted to said first port while said digital 
signal is being processed; and 

(h) applying a time stamp to said digital signal that is 
obtained from said first signal, said time stamp being 
indicative of when said first signal was received in step 
(b). 

wherein a quality contour map is generated based in 
part on: i) the respective time stamps for a plurality 
of said first signals received over a period of time, 
and ii) the respective quality scores generated in step 
(c), when step (b) is performed at a plurality of 
different locations at a plurality of different times 
within a region. 

2. A method for evaluating the quality of a transmission 
channel having a first port and a second port, comprising the 
steps of: 

(a) transmitting a first signal from said first port to said 
second port via said transmission channel, said first 
signal being an analog signal; 

(b) receiving said first signal at said second port, and 
converting said first signal to a digital signal; 

(c) analyzing said digital signal using voice recognition 
technology to generate a quality score within a prede- 
termined time frame; 

(d) monitoring step (c) to ensure that said quality score is 
generated during said predetermined time frame; 

(e) assigning a default quality score if a quality score is 
not generated during said predetermined time frame; 

(f) transmitting a second signal from said second port to 
said first port, via said transmission channel, after said 
quality score is either generated or assigned said default 
quality score; and 

(g) monitoring step (f) to ensure that said second signal is 
not being transmitted to said first port while said digital 
signal is being processed, 

wherein the quality score is calculated based on the 
equation: 

VQScoTC-SRCanfidenceFacloi* dM-dB, 
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wherein: 

VQScore is the quality score, 

SRConfidenceFactor is a confidence factor com- 
puted by a voice recognition software module that 
performs an initial voice recognition processing 
on said digital signal, 

dM is a multiplicative calibration factor, and 

dB is an additive calibration factor. 

3, An apparatus for evaluating the quality of a transmis- 
sion channel, said apparatus having an interface to said 
transmission channel, said apparatus comprising: 

(a) a processor configured to transmit a second signal via 
said transmission channel and to receive and process a 
first signal via said transmission channel, said first 
signal corresponding to a digitized version of an analog 
signal that was sent from another apparatus to said 
apparatus over said transmission channel, said proces- 
sor performing a digital-to-analog conversion of said 
first signal to obtain said first signal; 

(b) a voice recognition module associated with said 
processor for analyzing, using voice recognition 
technology, said first signal to generate a preliminary 
score; 

(c) a quality score module associated with said processor 
for generating a quality score based on said preliminary 
score during a predetermined time frame; 

(d) a monitoring module associated with said processor 
for ensuring that said quality score is generated during 
a predetermined time frame and for assigning a default 
quality score if a quality score is not generated during 
said predetermined time frame; 

(e) a synchronization module associated with said pro- 
cessor for transmitting said second signal via said 
transmission channel after generating said quality score 
or assigning said default quality score and after said 
first signal is processed; and 

(f) a time stamp applying unit configured to apply a time 
stamp to said first signal, said time stamp being indica- 
tive of when said first signal was received by said 
apparatus, 

wherein a quality contour map is generated based in 
part on: i) the respective time stamps for a plurality 
of said first signals received over a period of time, 
and ii) the respective quality scores generated for 
said plurality of first signals, and 

wherein said plurality of first signals are received at a 
plurality of different locations at a plurality of dif- 
ferent times within a region. 

4. An apparatus for evaluating the quality of a transmis- 
sion channel, said apparatus having an interface to said 
transmission channel, said apparatus comprising: 

(a) a processor configured to transmit a second signal via 
said transmission channel and to receive and process a 
first signal via said transmission channel, said first 
signal corresponding to a digitized version of an analog 
signal that was sent from another apparatus to said 
apparatus over said transmission channel, said proces- 
sor performing a digital-to-analog conversion of said 
first signal to obtain said first signal; 

(b) a voice recognition module associated with said 
processor for analyzing, using voice recognition 
technology, said first signal to generate a preliminary 
score; 

(c) a quality score module associated with said processor 
for generating a quality score based on said preliminary 
score during a predetermined time frame; 
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(d) a monitoring module associated with said processor 
for ensuring that said quality score is generated during 
a predetermined time frame and for assigning a default 
quality score if a quality score is not generated during 
said predetermined time frame; and 

(e) a synchronization module associated with said pro- 
cessor for transmitting said second signal via said 
transmission channel after generating said quality score 
or assigning said default quality score and after said 
first signal is processed, 

wherein the quality score is calculated by said quality 
score module based on the equation: 

70Score=£RConfidenceFactor*dM-<iB, 

wherein: 

VQScore is the quality score, 
SRConfidenceFactor is the preliminary score com- 
puted by said voice recognition module, 
dM is a multiplicative calibration factor, and 
dB is an additive calibration factor. 
5, A system for evaluating the quality of a transmission 
channel, comprising: 

(a) a first processor; 

(b) a second processor; 

(c) a transmission channel therebetween; said first pro- 
cessor and second processor configured to interface 
with said transmission channel and to transmit a second 
signal to the other and receive a first signal from the 
other via said transmission channel, said first and 
second signals being sent as analog signals over said 
transmission channel; 

(d) a first programming module associated with said first 
processor for: 1) analyzing, using voice recognition 
technology, a digital representation of said first signal 
to generate a preliminary score; 2) generating a quality 
score based on said preliminary score during a prede- 
termined time frame or assigning a default quality score 
if a quality score is not generated during said prede- 
termined time frame; and 3) transmitting said second 
signal via said transmission channel after either gener- 
ating said quality score or assigning said default score 
and after said digital representation of said first signal 
is processed; and 

(e) a second programming module associated with said 
second processor for: 1) analyzing, using voice recog- 
nition technology, a digital representation of said first 
signal to generate a preliminary score; 2) generating a 
quality score based on said preliminary score during a 
predetermined time frame or assigning a default quality 
score if a quality score is not generated during said 
predetermined time frame; and 3) transmitting said 
second signal via said transmission channel after either 
generating said quality score or assigning said default 
score and after said digital representation of said first 
signal is processed, 

wherein each of said first and second programming 
modules comprises: 

a time stamp applying unit configured to apply a time 
stamp to said first signal, said time stamp being 
indicative of when said first signal was received 
by said apparatus, 

wherein a quality contour map is generated based in 
part on: i) the respective time stamps for a plu- 
rality of said first signals received over a period of 
time, and ii) the respective quality scores gener- 
ated for said plurality of first signals, and 
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wherein said plurality of first signals are received at 
a plurality of different locations at a plurality of 
different times within a region. 

6. A system for evaluating the quality of a transmission 
channel, comprising: 5 

(a) a first processor; 

(b) a second processor; 

(c) a transmission channel therebetween; said first pro- 
cessor and second processor configured to interface Q 
with said transmission channel and to transmit a second 
signal to the other and receive a first signal from the 
other via said transmission channel, said first and 
second signals being sent as analog signals over said 
transmission channel; J5 

(d) a first programming module associated with said first 
processor for: 1) analyzing, using voice recognition 
technology, a digital representation of said first signal 
to generate a preliminary score; 2) generating a quality 
score based on said preliminary score during a prede- 2 o 
termined time frame or assigning a default quality score 

if a quality score is not generated during said prede- 
termined time frame; and 3) transmitting said second 
signal via said transmission channel after either gener- 
ating said quality score or assigning said default score 2 s 
and after said digital representation of said first signal 
is processed; and 

(e) a second programming module associated with said 
second processor for: 1) analyzing, using voice recog- 
nition technology, a digital representation of said first 30 
signal to generate a preliminary score; 2) generating a 
quality score based on said preliminary score during a 
predetermined time frame or assigning a default quality 
score if a quality score is not generated during said 
predetermined time frame; and 3) transmitting said 35 
second signal via said transmission channel after either 
generating said quality score or assigning said default 
score and after said digital representation of said first 
signal is processed, 

wherein the quality score is calculated based on the 40 
equation: 

VeScore-SRConfidenccFactor*dW-dB, 

wherein: 

VQScore is the quality score, 45 
SRConfidenceFactor is the preliminary score gener- 
ated by the voice recognition technology, 
dM is a multiplicative calibration factor, and 
dB is an additive calibration factor. 

7. A system for evaluating more than one transmission 50 
channel simultaneously, comprising: 

(a) a first processor; 

(b) a second processor; 

(c) a plurality of transmission channels therebetween; said 55 
first processor and second processor configured to 
interface with each of said plurality of transmission 
channels and to send a second signal to the other and 
receive a first signal from the other via each of said 
plurality of transmission channels, said first and second 60 
signals being sent as analog signals over said plurality 

of transmission channels; 

(d) a first programming module associated with said first 
processor for: 1) analyzing, using voice recognition 
technology, a respective digital representation of each 65 
of said first signals to generate a preliminary score for 
each first signal; 2) generating a quality score based on 
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said preliminary score during a predetermined time 
frame or assigning a default quality score if a quality 
score is not generated during said predetermined time 
frame; and 3) transmitting said second signal via each 
of said transmission channels after either generating 
said quality score or assigning said default score and 
after said respective digital representations of all of said 
first signals are processed; and 
(e) a second programming module associated with said 
second processor for: 1) analyzing, using voice recog- 
nition technology, a respective digital representation of 
each of said first signals to generate a preliminary score 
for each first signal; 2) generating a quality score based 
on said preliminary score during a predetermined time 
frame or assigning a default quality score if a quality 
score is not generated during said predetermined time 
frame; and 3) transmitting said second signal via each 
of said transmission channels after either generating 
said quality score or assigning said default score and 
after said respective digital representations of all of said 
first signals are processed, 

wherein each of said first and second programming 
modules comprises: 

a time stamp applying unit configured to apply a time 
stamp to said first signal, said time stamp being 
indicative of when said first signal was received 
by said apparatus, 

wherein a quality contour map is generated based in 
part on: i) the respective time stamps for a plu- 
rality of said first signals received over a period of 
time, and ii) the respective quality scores gener- 
ated for said plurality of first signals, and 

wherein said plurality of first signals are received at 
a plurality of different locations at a plurality of 
different times within a region. 
8. A system for evaluating more than one transmission 
channel simultaneously, comprising: 

(a) a first processor; 

(b) a second processor; 

(c) a plurality of transmission channels therebetween; said 
first processor and second processor configured to 
interface with each of said plurality of transmission 
channels and to send a second signal to the other and 
receive a first signal from the other via each of said 
plurality of transmission channels, said first and second 
signals being sent as analog signals over said plurality 
of transmission channels; 

(d) a first programming module associated with said first 
processor for: 1) analyzing, using voice recognition 
technology, a respective digital representation of each 
of said first signals to generate a preliminary score for 
each first signal; 2) generating a quality score based on 
said preliminary score during a predetermined time 
frame or assigning a default quality score if a quality 
score is not generated during said predetermined time 
frame; and 3) transmitting said second signal via each 
of said transmission channels after either generating 
said quality score or assigning said default score and 
after said respective digital representations of all of said 
first signals are processed; and 

(e) a second programming module associated with said 
second processor for: 1) analyzing, using voice recog- 
nition technology, a respective digital representation of 
each of said first signals to generate a preliminary score 
for each first signal; 2) generating a quality score based 
on said preliminary score during a predetermined time 
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frame or assigning a default quality score if a quality 
score is not generated during said predetermined time 
frame; and 3) transmitting said second signal via each 
of said transmission channels after either generating 
said quality score or assigning said default score and 5 
after said respective digital representations of all of said 
first signals are processed, 

wherein the quality score is calculated based on the 
equation: 

VGScorc«»5RConfidcnceFactor*(iA/-^ff, 10 

wherein: 

VQScore is the quality score, 
SRConfidenceFactor is the preliminary score gener- 
ated by the voice recognition technology, 
dM is a multiplicative calibration factor, and 15 
dB is an additive calibration factor. 

9. A system for evaluating the quality of a transmission 
channel, comprising: 

(a) a processor configured to interface to said transmission 
channel and to send a second signal and to receive a 20 
first signal via said transmission channel, said first 
signal corresponding to a digitized version of an analog 
signal that was sent from another apparatus to said 
apparatus over said transmission channel, said proces- 
sor performing a digital-to-analog conversion of said 25 
first signal to obtain said first signal, said second signal 
being sent as an analog signal over said transmission 
channel; 

(b) a communication device coupled to said transmission 
channel and configured to at least send said first signal 30 
to said processor via said transmission channel; 

(c) a voice recognition module associated with said pro- 
cessor for analyzing, using voice recognition 
technology, said first signal to generate a preliminary 
score; 35 

(d) a quality score module associated with said processor 
for generating a quality score based on said preliminary 
score during a predetermined time frame or assigning a 
default quality score if a quality score is not generated 
during said predetermined time frame; and 40 

(e) a time stamp applying unit configured to apply a time 
stamp to said first signal, said time stamp being indica- 
tive of when said first signal was received by said 
processor, 

wherein a quality contour map is generated based in 45 
part on: i) the respective time stamps for a plurality 
of said first signals received over a period of time, 
and ii) the respective quality scores generated for 
said plurality of first signals, and 

wherein said plurality of first signals are received at a 50 
plurality of different locations at a plurality of dif- 
ferent times within a region. 

10. A system for evaluating the quality of a transmission 
channel, comprising: 

(a) a processor configured to interface to said transmission 55 
channel and to send a second signal and to receive a 
first signal via said transmission channel, said first 
signal corresponding to a digitized version of an analog 
signal that was sent from another apparatus to said 
apparatus over said transmission channel, said proces- 60 
sor performing a digital-to-analog conversion of said 
first signal to obtain said first signal, said second signal 
being sent as an analog signal over said transmission 
channel; 

(b) a communication device coupled to said transmission 65 
channel and configured to at least send said first signal 

to said processor via said transmission channel; 
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(c) a voice recognition module associated with said pro- 
cessor for analyzing, using voice recognition 
technology, said first signal to generate a preliminary 
score; and 

(d) a quality score module associated with said processor 
for generating a quality score based on said preliminary 
score during a predetermined time frame or assigning a 
default quality score if a quality score is not generated 
during said predetermined time frame, 

wherein the quality score is calculated by said quality 
score module based on the equation: 

Vi2Score-5RConfidenceFactor*<i/Vf-£iB ( 
wherein: 

VQScore is the quality score, 
SRConfidenceFactor is the preliminary score com- 
puted by said voice recognition module, 
dM is a multiplicative calibration factor, and 
dB is an additive calibration factor. 

11. A system for evaluating more than one transmission 
channel simultaneously, comprising: 

(a) a processor; 

(b) a plurality of communication devices; and 

(c) a plurality of transmission channels therebetween; 
each of said plurality of communication devices con- 
figured to interface with one of said plurality of trans- 
mission channels and to send a first signal as an analog 
signal along said one of said plurality of transmission 
channels to said processor, said processor configured to 
interface with more than one of said plurality of trans- 
mission channels at any given time, to receive several 
first signals at once; said first signals being transmitted 
by each of said plurality of communication devices and 
to transmit an outgoing audio signal to each of said 
plurality of communication devices simultaneously; 

(d) a voice recognition module associated with said 
processor for analyzing, using voice recognition 
technology, a respective digital representation of each 
of said first signals to generate a preliminary score for 
each first signal; 

(e) a quality score module for generating a quality score 
based on said preliminary score during a predetermined 
time frame or assigning a default quality score if a 
quality score is not generated during said predeter- 
mined time frame; and 

(f) a time stamp applying unit configured to apply a time 
stamp to each of said first signals, said time stamp 
being indicative of when each of said first signals was 
received by said processor, 

wherein a quality contour map is generated based in 
part on: i) the respective time stamps for a plurality 
of said first signals received over a period of time, 
and ii) the respective quality scores generated for 
said plurality of first signals, and 

wherein said plurality of first signals are received at a 
plurality of different locations at a plurality of dif- 
ferent times within a region. 

12. A system for evaluating more than one transmission 
channel simultaneously, comprising: 

(a) a processor; 

(b) a plurality of communication devices; and 

(c) a plurality of transmission channels therebetween; 
each of said plurality of communication devices con- 
figured to interface with one of said plurality of trans- 
mission channels and to send a first signal as an analog 
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signal along said one of said plurality of transmission 
channels to said processor, said processor configured to 
interface with more than one of said plurality of trans- 
mission channels at any given time, to receive several 
first signals at once; said first signals being transmitted 5 
by each of said plurality of communication devices and 
to transmit an outgoing audio signal to each of said 
plurality of communication devices simultaneously; 

(d) a voice recognition module associated with said 
processor for analyzing, using voice recognition 10 
technology, a respective digital representation of each 

of said first signals to generate a preliminary score for 
each first signal; and 

(e) a quality score module for generating a quality score 
based on said preliminary score during a predetermined 
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time frame or assigning a default quality score if a 
quality score is not generated during said predeter- 
mined time frame, 

wherein the quality score is calculated by said quality 
score module based on the equation: 

VGScore^SKConfidcnceFactor*dA/-dB, 

wherein: 

VQScore is the quality score, 
SRConfidenceFactor is the preliminary score com- 
puted by said voice recognition module, 
dM is a multiplicative calibration factor, and 
dB is an additive calibration factor. 

* * * * * 
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